1. Field of the Invention
The present invention relates to a method for creating a mosaic made up of a plurality of small overlapping photographic images wherein obstructions due to moving foreground objects are minimized.
2. Related Art
Digital maps and digital map databases are used in navigation systems. Digital maps are obtained by various methods, including high resolution imagery from space, as well as orthorectified images taken from land-based mobile vehicles. In the latter case, the images obtained from land-based mapping systems must be converted to an orthorectified image which is scale-corrected and depicts ground features as seen from above in their exact ground positions. An orthorectified image is a kind of aerial photograph that has been geometrically corrected such that the scale of the photograph is uniform, meaning that the photograph can be considered equivalent to a map. An orthorectified image can be used to measure true distances, because it is an accurate representation of the surface of interest, e.g., the Earth's surface. Orthorectified images are adjusted for topographic relief, lens distortion and camera tilt.
Orthorectified images can be obtained very efficiently from aerial images. However, errors are often introduced, which can result in inaccurate mapping of the geo-positioned data. One problem is that normally aerial images are not taken exactly perpendicular to the surface of the Earth. Even when a picture is taken close to perpendicular, only its exact center line will be perpendicular. In order to orthorectify such an image, height of terrain information must be additionally obtained. The lack of accurate height information of objects in an aerial image, in combination with the triangulation process used to determine the orthorectified image, can result in an inaccuracy of such images up to a dozen meters or more. The accuracy can be improved by taking overlapping images and comparing the same surface obtained from subsequent images. However, there is a limit to the accuracy obtained from this method as compared to its cost.
Herein, the term “horizontal” data or information corresponds to objects that have a surface parallel or substantially parallel to the Earth's surface. The term “vertical” data or information corresponds to objects that can be seen with a looking axis generally parallel to the Earth's surface. Vertical information cannot be obtained from a typical look-down aerial or satellite image.
Mobile mapping vehicles, typically terrestrial based vehicles such as a van or car, but possibly also aerial vehicles, are used to collect mobile data for enhancement of digital map databases. The mobile mapping vehicles are typically fitted with a number of cameras, possibly some of them stereographic and all of them accurately geo-positioned as a result of having precision GPS and other position and orientation determination equipment (e.g., inertial navigation system—INS) on board. While driving the road network or an established course, the geo-coded image sequences are captured in successive frames or images. Geo-coded means that a position, computed by the GPS receiver and possibly INS, and possibly additional heading and/or orientation data associated with the image, is attached to the metadata of each image captured by the camera. The mobile mapping vehicles record more than one image sequence of the surface of interest, e.g., a road surface, and for each image of an image sequence, the geo-position in a geographic coordinate reference system is accurately determined together with the position and orientation data of the image sequence with respect to the geo-position. Image sequences with corresponding geo-position information are referred to as geo-coded image sequences. Other data may also be collected by other sensors, simultaneously and similarly geo-coded.
Prior techniques for obtaining orthorectified tiles for use in assembling a bird's eye mosaic (BEM) of a large surface of interest, such as the Earth, are known. An excellent example of this technique is described in the Applicant's International Publication No. WO/2008/044927, published Jul. 17, 2008. In jurisdictions where incorporation by reference is recognized, the entire disclosure of the said International Publication is hereby incorporated by reference and relied upon.
According to known techniques, orthorectified images are assembled together to create a mosaic without considering the quality of the image content contained therein. Rather, such images are typically tiled in sequence, one after the other, much like shingles are overlapped one upon another in courses on a roof. While generally effective, it is often the case that moving objects captured in the photograph images, e.g., motor vehicles that are passing or being passed by the mobile mapping vehicle, appear in an overlaying tile but not an underlaying tile, such that a less desirable tile is overlaid upon a more desirable tile. As a result, a moving foreground object that partially obscures the map of the road surface may appear on the completed BEM.
The Applicant's copending application P6015247 PCT, titled Method Of An Apparatus For Producing A Multi-Viewpoint Panorama, describes a method to generate vertical panoramas using a sequence of vertical images taken from multiple viewpoints from a mobile mapping vehicle. While creating a panorama, objects close to the camera are detected using laser scanner data. Objectionable objects captured in the image are removed by marking the parts in the vertical images that should not be used. The parts that should be used are then projected onto the panorama surface.
The use of laser data, particularly in connection with vertical images, is an expensive, cumbersome and less desirable technique for creating orthorectified horizontal images for use in creating a bird's eye mosaic (BEM). According, there is a need for identifying moving foreground objects in an orthorectified photographic image of a surface of interest which does not rely upon the use of laser scanners or other cumbersome techniques, particularly when existing image data may be available without contemporaneous laser scanning data.